Self-supervised Log Parsing

Logs are extensively used during the development and maintenance of software systems. They collect runtime events allow tracking code execution, which enables a variety critical tasks such as troubleshooting fault detection. However, large-scale systems generate massive volumes semi-structured log records, posing major challenge for automated analysis. Parsing records with free-form text messages into structured templates is first crucial step that further Existing approaches rely on log-specific heuristics or manual rule extraction. These often specialized in parsing certain types, thus, limit performance scores generalization. We propose novel technique called NuLog utilizes self-supervised learning model formulates task masked language modeling (MLM). In process parsing, extracts summarizations from logs form vector embedding. This allows coupling MLM pre-training downstream anomaly detection task. evaluate 10 real-world datasets compare results 12 techniques. The show outperforms existing methods accuracy an average 99% achieves lowest edit distance to ground truth templates. Additionally, two case studies conducted demonstrate ability approach log-based both supervised unsupervised scenario. can be successfully support tasks. implementation available at https://github.com/nulog/nulog.